Moving picture encoding/transmission system, moving picture encoding/transmission method, and encoding apparatus, decoding apparatus, encoding method decoding method and program usable for the same

ABSTRACT

An object of the present invention is to allow portions which are subjected to a coding in inter-frame prediction mode and portions which are subjected to coding in intra-frame prediction mode to be mixed in one macro block without changing the framework of macro blocks. The present invention provides an encoder which encodes each of first image blocks of a video. The encoder includes prediction mode selection information generating means for generating prediction mode selection information which indicates that a first prediction mode for reducing temporal redundancy is applied to each of second image blocks or that a second prediction mode for reducing spatial redundancy is applied to each of the second image blocks. The second image blocks are obtained by dividing the first image blocks. The encoder includes predictive residual signal generating means for generating a predictive residual signal by applying the selected first or second prediction mode to each of the second image blocks. The encoder includes transmitting means for transmitting the prediction mode selection information in association with the predictive residual signal.

TECHNICAL FIELD

[0001] The present invention relates to a video coding transmissionsystem and a video coding transmission method, and relates to an encode,a decoder, an encoding method, a decoding method, and a program whichare suitable for use in the same.

BACKGROUND ART

[0002] A description will be given of examples of an encoder and adecoder used in a conventional video coding transmission system withreference to FIGS. 1 and 2. FIG. 1 is a block diagram showing aschematic configuration of the encoder used in the conventional videocoding transmission system. FIG. 2 is a block diagram showing aschematic configuration of the decoder used in the conventional videocoding transmission system.

[0003] The encoder 120 shown in FIG. 1 and the decoder 140 shown in FIG.2 are a digital video encoder and a digital video decoder, respectively,which are compliant with the H.263 coding system described in ITU-TRecommendation H.263 “Video coding for low bit rate communication”.

[0004] The encoder 120 reduces temporal redundancy by the motioncompensated inter-frame prediction and reduces spatial redundancy byorthogonal transformation (for example, DCT: Discrete Cosine Transform),so as to perform information compressing and encoding for an input videosignal 2 as a digital video.

[0005] An inputting section 121 receives the input video signal 2,namely, a time sequence of frame images. Herein, in the encoder 120, aframe image which is now being encoded is referred to as a “currentframe”.

[0006] The inputting section 121 divides the “current frame” into squareregions (“macro blocks (first image blocks)”) of 16×16 pixels andsequentially sends the “macro blocks” to a motion estimating section 122and a subtracting section 124. Herein, a “macro block” which is nowbeing encoded is referred to as a “current macro block”.

[0007] The motion estimating section 122 estimates “motion vectors” anddetermines “macro block modes (described later)” on a macro block basis.

[0008] The motion estimating section 122 finds out a portion (“motionprediction data”) similar to the current macro block in a predeterminedsearch area of a frame image (referred to as a “reference frame”) whichwas encoded in past and stored in a frame memory 132, and estimates anamount of two-dimensional spatial motion from the current macro block to“motion prediction data” as a “motion vector”.

[0009] For example, the motion estimating section 122 can perform theaforementioned estimation of “motion vectors” using a “block matching”.Specifically, the motion estimating section 122 sets the search areaaround a spatial position of the current macro block in the “referenceframe” within the frame memory 132, and calculates the “sum of squaresof differences” or the “sum of absolute differences” between image datawithin the search area and the current macro block. The motionestimating section 122 then obtains image data that minimizes thecalculated “sum of squares of differences” or “sum of absolutedifferences” within the search area, as the “motion prediction data”.And the motion estimating section 122 estimates the amount oftwo-dimensional spatial motion from the current macro block to the“motion prediction date” as the “motion vector”.

[0010] The motion estimating section 122 sends the estimated “motionvector” to a motion compensating section 123 and a variable lengthencoding section 127.

[0011] The motion estimating section 122 determines a “macro block mode”applied to the current macro block. Herein, the “macro block mode”indicates a method (“prediction mode”, the number of motion vectors,eta.) of generating a “predictive residual signal (described later)” forthe current macro block. As shown in FIG. 3, the macro block modeincludes the “INTRA mode (the intra-frame prediction mode is applied)”,“INTER mode (the inter-frame prediction mode is applied)”, and “INTER 4Vmode (the inter-frame prediction mode with four motion vectors isapplied)”.

[0012] The “prediction mode” indicates application of the “inter-frameprediction mode” which reduces the temporal redundancy or application ofthe “intra-frame prediction mode” which reduces the spatial redundancyfor the current macro block.

[0013] Specifically, the motion estimating section 122 selects a “macroblock mode” that minimizes the power of the predictive residual signal(described later) among the “INTRA mode”, “INTER mode”, and “INTER 4Vmode” based on the estimated “motion vector”.

[0014] The motion estimating section 122 sends the determined “macroblock mode” to the motion compensating section 123 and the variablelength encoding section 127.

[0015] The motion compensating section 123 sends “control information”obtained based on the “macro block mode” and the “motion vector” sentfrom the motion estimating section 122, to the subtracting section 124.

[0016] For example, when the “macro block mode” sent from the motionestimating section 122 is the “INTRA mode”, the motion compensatingsection 123 notifies the subtracting section 124 of only the received“macro block mode (INTRA mode)” as the “control information” withoutforming a “predicted image block (described later)”, namely, withoutperforming the motion compensated inter-frame prediction for the“current macro block”.

[0017] When the “macro block mode” received from the motion estimatingsection 122 is the “INTER mode” or “INTER 4V mode”, the motioncompensating section 123 performs the motion compensated Inter-frameprediction for the current macro block using the “motion vector” sentfrom the motion estimating section 122 and the reference frame stored inthe frame memory 132, so as to form the “predicted image block”.

[0018] Herein, in the “INTER mode”, one motion vector is assigned to amacro block of 16×16 Pixels. In the “INTER 4V mode”, one motion vectoris assigned to a sub-block of 8×8 pixels.

[0019] The motion compensating section 123 sends the “predicted imageblock”, the “macro block mode”, and the “motion vector” to thesubtracting section 124 as the “control information”. Moreover, themotion compensating section 123 sends the “predicted image block” to anadding section 131.

[0020] The subtracting section 124 sends predetermined information to anorthogonal transforming section 125 according to the “controlinformation” sent by the motion compensating section 123.

[0021] Specifically, when the “macro block mode” is the “INTER mode” or“INTER 4V mode”, the subtracting section 124 reduces the temporalredundancy between temporally consecutive macro blocks, by obtaining adifference between the “current macro block” sent from the inputtingsection 121 and the “predicted image block” sent from the motioncompensating section 123.

[0022] Herein, the difference obtained by the subtracting section 124 isreferred to as the “predictive residual signal”. The subtracting section124 sends this “predictive residual signal” to the orthogonaltransforming section 125.

[0023] When the “macro block mode” is the “INTRA mode”, the subtractingsection 124 sends the “current macro block” sent from the inputtingsection 124, to the orthogonal transforming section 125, because the“predictive residual signal” for the “predicted image block” is not sentfrom the motion compensating section 123.

[0024] The orthogonal transforming section 125 reduces the spatialredundancy within the “predictive residual signal”, by performingorthogonal transformation (for example. DCT) in sub-blocks of 8×8 pixelsfor the “predictive residual signal” sent from the subtracting section124.

[0025] The orthogonal transforming section 125 sends “orthogonaltransformation coefficients (for example, DCT coefficients)” obtained bythe orthogonal transformation, to a quantizing section 126.

[0026] The quantizing section 126 quantizes the “orthogonaltransformation coefficients” sent from the orthogonal transformingsection 125. The quantizing section 126 then sends the “quantizedorthogonal transformation coefficients” obtained by the quantization, tothe variable length encoding section 127 and a dequantizing section 129.

[0027] The variable length encoding section 127 performs variable lengthencoding for the “quantized orthogonal transformation coefficients” sentby the quantizing section 126, and the “motion vector” and “macro blockmode” sent from the motion estimating section 122, and multiplexes thesame with a compressed bit stream 3. The variable length encodingsection 127 sands the compressed bit stream 3 to an outputting section128.

[0028] The outputting section 128 transmits the compressed bit stream 3constituting one or a plurality of frame images sent from the variablelength encoding section 127, to a network 1.

[0029] The dequantizing section 129 dequantizes the “quantizedorthogonal transformation coefficients” sent by the quantizing section126, and sends the obtained “orthogonal transformation coefficients” toan inverse orthogonal transforming section 130.

[0030] The inverse orthogonal transforming section 130 performs inverseorthogonal transformation (for example, inverse DCT) for the “orthogonaltransformation coefficients” sent by the dequantizing section 129, andsends the “predictive residual signal” obtained by the inverseorthogonal transformation, to the adding section 131,

[0031] The adding section 131 sends the result of adding up the“predicted image block” sent by the motion compensating section 123 andthe “predictive residual signal” sent by the inverse orthogonaltransforming section 130, to the frame memory 132.

[0032] When the “INTRA mode” is selected as the “macro block mode”, theadding section 131 sends the “predictive residual signal sent by theinverse orthogonal transforming section 130 (the current macro blocksent by the inputting section 121)” to the frame memory 132, because the“predicted image block” is not generated by the motion compensatingsection 123 (motion compensated inter-frame prediction is notperformed).

[0033] The frame memory 132 constructs and stores the “reference frame”based on the information sent by the adding section 131 (the currentmacro block). The frame memory 132 sends the “reference frame” to themotion estimating section 122 and the motion compensating section 123.

[0034] The decoder 140 shown in FIG. 2 reproduces an output video signal4 from the compressed bit stream 3 sent by the encoder 120.

[0035] An inputting section 141 receives the compressed bit stream 3,and sends the same to a variable length decoding section 142.

[0036] The variable length decoding section 142 decodes the “quantizedorthogonal transformation coefficients”, the “motion vector”, and the“macro block mode” for each macro block starting from the head of eachframe image in the compressed bit stream 3 sent by the inputting section141.

[0037] The variable length decoding section 142 sends the decoded“quantized orthogonal transformation coefficients” to a dequantizingsection 143. When the “macro block mode” is the “INTER mode” or “INTER4V mode”, the variable length decoding section 142 sends the one orseveral decoded “motion vectors” and “macro block mode” to a motioncompensating section 145.

[0038] The dequantizing section 143 dequantizes the “quantizedorthogonal transformation coefficients” sent by the variable lengthdecoding section 142, so as to obtain the “orthogonal transformationcoefficients” and to send the obtained “orthogonal transformationcoefficients” to an inverse orthogonal transforming section 144.

[0039] The inverse orthogonal transforming section 144 performs inverseorthogonal transformation for the “orthogonal transformationcoefficients” sent by the dequantizing section 143, so as to obtain the“predictive residual signal” and to send the obtained “predictiveresidual signal” to the adding section 146.

[0040] The motion compensating section 145 generates the “predictedimage block” based on the reference frame stored in a frame memory 147and the “motion vector” and “macro block mode” sent by the variablelength decoding section 142, and sends the generated “predicted imageblock” to an adding section 146.

[0041] The adding section 146 adds up the “predictive residual signal”sent by the inverse orthogonal transforming section 144 and the“predicted image block” sent by the motion compensating section 145, soas to generate a macro block constituting the output video signal 4 andto send the generated macro block to an outputting section 148.

[0042] However, when the “macro block mode” is the “INTRA mode”, theadding section 146 sends the “predictive residual signal” sent by theinverse orthogonal transforming section 144 to the outputting section148 as the macro block constituting the output video signal 4, becausethe “predicted image block” is not sent by the motion compensatingsection 145.

[0043] The frame memory 147 constructs and stores the “reference frame”based on the information sent by the adding section 146 (the macroblock). The frame memory 147 sends the “reference frame” to the motioncompensating section 145.

[0044] The outputting section 148 constructs the output video signal 4based on the information sent by the adding section 146 (the macroblock), and outputs the output video signal 4 to a display device (notshown) at a predetermined timing of display.

[0045] As described above, in the conventional video coding transmissionsystem, the “macro block mode” is determined for each macro block andthe “coding information (motion vector, quantization parameter, eta.)set for each macro block is shared for coding process, thus increasingthe coding efficiency.

[0046] However, the conventional video coding transmission system cannotset a plurality of the “macro block mode” in one macro block.Accordingly, there was a problem in that efficient coding cannot beperformed when one macro block includes a portion (bird portion) whichshould be subjected to coding with the “intra-frame prediction mode” anda portion (cloud portion) which should be subjected to coding in the“inter-frame prediction mode”, as shown in FIG. 4.

[0047] In order to solve this problem, there is a method in which themacro blocks are made smaller and the unit for switching the selectionbetween the “macro block modes” is reduced. However, this methodincreases the number of macro blocks and then increases the number oftransmissions of the coding information of each macro block necessaryfor coding, which causes a problem that reduces the coding efficiency.

[0048] Therefore, the present invention was made in the light of theaforementioned problems, and an object thereof is to switch the “macroblock modes”, so as to allow the portion which is subjected to codingwith the “intra-frame prediction mode” and the portion which issubjected to coding with the “inter-frame prediction mode” to be mixedin one macro block without changing the size and framework of macroblocks.

DISCLOSURE OF THE INVENTION

[0049] A first feature of the present invention is summarized as anencoder for encoding each of first image blocks of a video. The encoderincludes prediction mode selection information generating means,predictive residual signal generating means, and transmitting means. Theprediction mode selection information generating means generatesprediction mode selection information which indicates that a firstprediction mode for reducing temporal redundancy is applied to each ofsecond image blocks or that a second prediction mode for reducingspatial redundancy is applied to each of the second image blocks. Thesecond image blocks are obtained by dividing the first image block. Thepredictive residual signal generating means generates a predictiveresidual signal by applying the selected first or second prediction modeto each of the second image blocks. The transmitting means encodes andtransmits the prediction mode selection information and the predictiveresidual signal.

[0050] In the first feature of the present invention, it is preferablethat the transmitting means encodes and transmits prediction moderelated information necessary for a coding process in the first orsecond prediction mode.

[0051] In addition, in the first feature of the present invention, it ispreferable that the predictive residual signal generating meansgenerates the predictive residual signal from each of the second imageblocks by motion compensated Inter-frame prediction using a motionvector, when the first prediction mode is selected to be applied, andthe predictive residual signal generating means generates the predictiveresidual signal from each of the second image blocks by intra-frameprediction when the second prediction mode is selected to be applied.

[0052] Moreover, in the first feature of the present invention, it ispreferable that the transmitting means encodes and transmits informationindicating the motion vector as the prediction mode related information,when the first prediction mode is selected to be applied.

[0053] Further, in the first feature of the present invention, it ispreferable that the predictive residual signal generating meansgenerates the predictive residual signal by motion compensatedinter-frame prediction using a same motion vector within the first imageblock.

[0054] Still further, the first feature of the present invention, it ispreferable that the transmitting means transmits the prediction modeselection information before the prediction mode related information.

[0055] Furthermore, in the first feature of the present invention, it ispreferable that the predictive residual signal generating meansgenerates the predictive residual signal from each of the second imageblocks by a pixel value prediction method using a value of a pixeladjacent to each of the second image blocks, when the second predictionmode is selected to be applied, and the transmitting means encodes andtransmits the pixel value prediction method an the prediction moderelated information.

[0056] Moreover, in the first feature of the present invention, it ispreferable that the transmitting means encodes and transmits theprediction mode related information in association with the predictionmode selection information.

[0057] Further, in the first feature of the present invention, it ispreferable that the transmitting means does not transmit the prediction,mode related information of the first prediction mode, when the firstprediction mode is not applied to each of the second image blocks

[0058] A second feature of the present invention is summarized as adecoder for decoding a video. The decoder includes prediction modeselection information decoding means and video decoding means. Theprediction mode selection information decoding means decodes predictionmode selection information which indicates that a first prediction modefor reducing temporal redundancy is applied to each of second imageblocks or that a second prediction mode for reducing spatial redundancyis applied to each of the second image blocks. The second image blocksare obtained by dividing a first image block. The first blocks areobtained by dividing the video. The video decoding means decodes each ofthe second image blocks of the video based on the first or secondprediction mode selected by the prediction mode selection information.

[0059] In the second feature of the present invention, it is preferablethat the decoder further includes prediction mode related informationdecoding means for decoding prediction mode related informationnecessary for a coding process in the first or second prediction mode,and the video decoding means decodes the video using the prediction moderelated information.

[0060] In addition, in the second feature of the present invention, itis preferable that the video decoding means decodes each of the secondimage blocks of the video from the predictive residual signal by motioncompensated inter-frame prediction, when the first prediction mode isselected to be applied by the prediction mode selection information, andthe video decoding means decodes each of the second image blocks of thevideo from the predictive residual signal by intra-frame prediction,when the second prediction mode is selected to be applied by theprediction mode selection information.

[0061] Further, in the second feature of the present invention, it ispreferable that the prediction mode related information decoding meansdecodes information indicating a motion vector as the prediction moderelated information, when the first prediction mode is selected to beapplied by the prediction mode selection information.

[0062] Still further, in the second feature of the present invention, itis preferable that the prediction mode selection information decodingmeans decodes the prediction mode selection information before theprediction mode related information.

[0063] Furthermore, in the second feature of the present invention, itis preferable that the video decoding means decodes the video usingmotion compensated inter-frame prediction by a same motion vector withinthe first image block, when the first prediction mode is selected to beapplied by the prediction mode selection information.

[0064] Moreover, in the second feature of the present invention, it ispreferable that the prediction mode related information decoding meansdecodes a pixel value prediction method related to the second imageblock as the prediction mode related information, when the secondprediction mode is selected to be applied by the prediction modeselection information, and the video decoding means decodes the videousing the pixel value prediction method.

[0065] Further, in the second feature of the present invention, it ispreferable that the prediction mode selection information decoding meansdecodes the prediction mode selection information of each of the secondimage blocks.

[0066] Still further, in the second feature of the present invention, itis preferable that the prediction mode selection information is encodedin association with the prediction mode related information.

[0067] Furthermore, in the second feature of the present invention, itis preferable that the prediction mode related information decodingmeans does not decode the prediction mode related information of thefirst prediction mode, when the prediction mode selection informationindicates that the first prediction mode is not applied to each of thesecond image blocks.

[0068] A third feature of the present invention is summarized as anencoding method of encoding each of first image blocks of a video. Theencoding method includes a step A of generating prediction modsselection information which indicates that a first prediction mode forreducing temporal redundancy is applied to each of second image blocksor that a second prediction mode for reducing spatial prediction mode isapplied to each of the second image blocks. The second image blocks areobtained by dividing a first image block, and the first image blocks areobtained by dividing the video. The encoding method includes a step B ofgenerating a predictive residual signal by applying the selected firstor second prediction mode to each of the second image blocks. Theencoding method includes a step C of encoding and transmitting theprediction mode selection information and the predictive residualsignal.

[0069] In the third feature of the present invention, it is preferablethat prediction mode related information necessary for a coding processin the first or second prediction mode is encoded and transmitted, inthe step C.

[0070] In addition, in the third feature of the present invention, it ispreferable that the predictive residual signal is generated from theeach of the Second image blocks by motion compensated inter-frameprediction using a motion vector when the first prediction mode isselected to be applied in the step B, and the predictive residual signalis generated from the each of the second image blocks by intra-frameprediction when the second prediction mode is selected to be applied inthe step B.

[0071] Further, in the third feature of the present invention, it inpreferable that information indicating the motion vector is encoded andtransmitted as the prediction mode related information when the firstprediction mode is selected to be applied, in the step C.

[0072] Still further, in the third feature of the present invention, itis preferable that the predictive residual signal is generated by“motion compensated inter-frame prediction using a same motion vectorwithin the first image blocks in the step B.

[0073] Furthermore, in the third feature of the present invention, it ispreferable that the prediction mode selection information is transmittedbefore the prediction mode related information, in the step C.

[0074] Moreover, in the third feature of the present invention, it ispreferable that the predictive residual signal is generated from each ofthe second image blocks by a pixel value prediction method using a valueof a pixel adjacent to each of the second image blocks when the secondprediction mode is selected to be applied in the step B, and the pixelvalue prediction method is encoded and transmitted an the predictionmode related information in the step C.

[0075] Further, in the third feature of the present invention, it ispreferable that the prediction mode related information is encoded andtransmitted in association with the prediction mode selectioninformation, in the step C.

[0076] Furthermore, in the third feature of the present invention, it ispreferable that the prediction mode related information of the firstprediction mode is not transmitted, when the first prediction mode isnot applied to each of the second image blocks, in the step C.

[0077] A fourth feature of the present invention is summarized as adecoding method for decoding a video. The decoding method includes astep A of decoding prediction mode selection information which indicatesthat a first prediction mode for reducing temporal redundancy is appliedto each of second image blocks or that a second prediction mode forreducing spatial prediction mode is applied to each of the second imageblocks. The second image blocks are obtained by dividing a first imageblock, and the first image blocks are obtained by dividing the video.The decoding method includes a step B of decoding each of the secondimage blocks of the video, based on the first or second prediction modeselected by the prediction mode selection information.

[0078] In the fourth feature of the present invention it is preferablethat prediction mode related information necessary for a coding processin the first or second prediction mode is decoded in the step A, and thevideo is decoded using the prediction mode related information, in thestop B.

[0079] Further, in the fourth feature of the present invention, it ispreferable that the video is decoded using motion compensatedinter-frame prediction in the step B, when the first prediction mode isselected to be applied by the prediction mode selection information inthe step A, and the video is decoded using intra-frame prediction in thestep B, when the second prediction mode is selected to be applied by theprediction mode selection information in the step A.

[0080] Still further, in the fourth feature of the present inventions itis preferable that information indicating A motion vector is decoded asthe prediction mode related information, when the first prediction modeis selected to be applied by the prediction mode selection information,in the step A.

[0081] Furthermore, in the fourth feature of the present invention it ispreferable that the prediction mode selection information is decodedbefore the prediction mode related information, in the Step A.

[0082] Moreover, in the fourth feature of the present invention, it ispreferable that the video is decoded using motion compensatedinter-frame prediction by a same motion vector within the first imageblock, when the first prediction mode is selected to be applied by theprediction mode selection information, in the step B.

[0083] Further, in the fourth feature of the present invention, it ispreferable that a pixel value prediction method related to the secondimage block to decoded as the prediction mode related information whenthe second prediction mode is selected to be applied by the predictionmode selection information in the step A, and the video is decoded usingthe pixel value prediction method in the step B.

[0084] Still further, in the fourth feature of the present invention, itis preferable that the prediction mode selection information of each ofthe second image blocks is decoded, in the step A.

[0085] Furthermore, in the fourth feature of the present invention, itis preferable that the prediction mode selection information is encodedin association with the prediction mode related information.

[0086] Moreover, in the fourth feature of the present invention, it ispreferable that the prediction mode related information of the firstprediction mode is not decoded, when the prediction mode selectioninformation indicates that the first prediction mode is not applied toeach of the second image blocks, in the step A.

[0087] A fifth feature of the present invention is summarized as aprogram causing a computer to function as an encoder which encodes eachfirst image block of a video. The program includes prediction modeselection information generating means, predictive residual signalgenerating means, and transmitting means. The prediction mode selectioninformation generating means generates prediction mode selectioninformation which indicates that a first prediction mode for reducingtemporal redundancy is applied to each of second image blocks or that asecond prediction mode for reducing spatial redundancy is applied toeach of the second image blocks. The second image blocks are obtained bydividing the first image block. The predictive residual signalgenerating means generates a predictive residual signal by applying theselected first or second prediction mode to each of the second imageblocks. The transmitting means encodes and transmits the prediction modeselection information and the predictive residual signal.

[0088] A sixth feature of the present invention is summarized as aprogram causing a computer to function as a decoder which decodes avideo. The program includes prediction mode selection informationdecoding means and video decoding means. The prediction mode selectioninformation decoding means decodes prediction mode selection informationwhich indicates that a first prediction mode for reducing temporalredundancy is applied to each of second image blocks or that a secondprediction mode for reducing spatial redundancy is applied to each ofthe second image blocks. The second image blocks are obtained bydividing a first image block, and the first image blocks are obtained bydividing the video. The video decoding means decodes each of the secondimage blocks of the video, based on the first or second prediction modeselected by the prediction mode selection information.

[0089] A seventh feature of the present invention is summarised as avideo coding transmission system having an encoder and a decoder. Theencoder includes prediction mode selection information generating means,predictive residual signal generating means, and transmitting means. Theprediction mode selection information generating means generatesprediction mode selection information which indicates that a firstprediction mode for reducing temporal redundancy is applied to each ofsecond image blocks or that a second prediction mode for reducingspatial redundancy is applied to each of the second image blocks. Thesecond image blocks are obtained by dividing a first image block, andthe first image blocks are obtained by dividing the video. Thepredictive residual signal generating means generates a predictiveresidual signal by applying the selected first or second prediction modeto each of the second image blocks. The transmitting means encodes andtransmits the prediction mode selection information and the predictiveresidual signal. The decoder includes prediction mode selectioninformation decoding means and video decoding means. The prediction modeselection information decoding means decodes the prediction modeselection information of each of the second image blocks. The videodecoding means decodes each of the second image blocks of the videobased on the first or second prediction mode selected by the predictionmode selection information.

[0090] An eighth feature of the present invention is summarized toinclude a step of encoding and transmitting prediction mode selectioninformation and a predictive residual signal in a coding process, a stepof decoding the prediction mode selection information of each of thesecond image blocks in a decoding process, and a step of decoding eachof the second image blocks of the video based on the first or secondprediction mode selected by the prediction mode selection information.The prediction mode selection information indicates that a firstprediction mode for reducing temporal redundancy is applied to each ofsecond image blocks or that a second prediction mode for reducingspatial redundancy is applied to each of the second image blocks. Thepredictive residual signal are generated by applying the selected firstor second prediction mode to each of the second image blocks. The secondimage blocks are obtained by dividing each of the first image blocks.

BRIEF DESCRIPTION OF THE DRAWINGS

[0091]FIG. 1 is a block diagram showing the configuration of an encoderused in a video coding transmission system according to a prior art.

[0092]FIG. 2 is a block diagram showing the configuration of a decoderused in the video coding transmission system according to the prior art.

[0093]FIG. 3 is a view showing macro block codes.

[0094]FIG. 4 is a view showing a case where a portion which is desirableto be encoded with an intra-frame prediction mode and a portion which isdesirable to be encoded with an inter-frame prediction mode are mixed ina macro block.

[0095]FIG. 5 is a schematic block diagram of a video coding transmissionsystem according to an embodiment of the present invention.

[0096]FIG. 6 is a block diagram showing the configuration of an encoderused in the video coding transmission system according to the embodimentof the present invention.

[0097]FIG. 7 is a block diagram showing the configuration of a decoderused in the video coding transmission system according to the embodimentof the present invention.

[0098] FIGS. 8(a) and 8(b) are tables showing definitions of macro blockmodes and block layer syntax which are extended in the video codingtransmission system according to the embodiment of the presentinvention.

[0099] PIG. 9 is a flowchart showing processing in a hybrid predictingsection of the encoder used in the video coding transmission systemaccording to the embodiment of the present invention.

[0100]FIG. 10 is a block diagram showing the configuration of an encoderused in a video coding transmission system according to an embodiment ofthe present invention.

[0101]FIG. 11 is a block diagram showing the configuration of a decoderused in the video coding transmission system according to the embodimentof the present invention.

[0102]FIG. 12 is a view showing motion estimation block Which are usedin a motion estimating section of the encoder used in the video codingtransmission system according to the embodiment of the presentinvention.

[0103]FIG. 13 is a view showing a principle of intra-frame pixel valueprediction performed by a spatial predicting section of the encoder usedin the video coding transmission system according to the embodiment ofthe present invention.

[0104] FIGS. 14(a) and 14(b) are tables showing definitions of macroblock modes and block layer syntax which are extended in the videocoding transmission system according to the embodiment of the presentinvention.

[0105]FIG. 15 is a flowchart showing processing in a hybrid predictingsection of the encoder used in the video coding transmission systemaccording to the embodiment of the present invention.

[0106] FIGS. 16(a) and 16(b) are views showing definitions of macroblock syntax used :n the video coding transmission system according tothe prior art and the embodiment of the present invention, respectively.

[0107]FIG. 17 is a table showing definitions of macro block syntaxextended in the video coding transmission system according to anembodiment of the present invention.

[0108]FIG. 18 is a flowchart showing processing in the hybrid predictingsection of the encoder used in the video coding transmission systemaccording to the embodiment of the present invention.

[0109]FIG. 19 is a view showing computer-readable recording media whichrecord a program functioning as the video coding transmission systemaccording to the present invention.

BEST MODES FOR CARRYING OUT THE PRESENT INVENTION

[0110] Hereinafter, a description will be given of embodiments of thepresent invention with reference to the drawings.

CONFIGURATION OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 1 OF THE PRESENT INVENTION

[0111] A description will be given of the configuration of a videocoding transmission system according to Embodiment 1 of the presentinvention with reference to the drawings. FIG. 5 is a block diagramshowing the configuration of the video coding transmission systemaccording to this embodiment.

[0112] In the video coding transmission system according to thisembodiment, an encoder 20 performs information compression and encodingprocess for an input video signal a which is a digital video, so as toobtain a “compressed bit stream 3”, and transmits the compressed bitstream 3 to a decoder 40 via a network 1. In this video codingtransmission system, the decoder 40 restores the compressed bit stream 3so as to obtain an output video signal 4.

[0113] Herein, the network 1 is assumed to be various types of meanswhich can receive and transmit the compressed bit stream 3, and isassumed to be, for example, a low-bit-rate digital communication line orthe like. Note that the video coding transmission system according tothis embodiment can be constructed using, for example, a system with adata storage device interposed instead of the communication line as thenetwork 1.

[0114] As shown in FIG. 5, the video coding transmission systemaccording to this embodiment includes the encoder 20 and the decoder 40.The video coding transmission system according to this embodiment iscompliant with the aforementioned H.263 coding system and has aconfiguration to which modifications necessary for carrying out thepresent invention are applied on that basis.

[0115] The encoder 20 used in the video coding transmission systemaccording to the embodiment is shown in FIG. 2. The encoder 20 accordingto this embodiment encodes each first image block (each macro block) ofvideos (input video signal 2).

[0116] Hereinafter, a description will be mainly given of differencesbetween the encoder 20 according to this embodiment and the encoder 120compliant with the conventional H.263 coding system. The encoder 20according to this embodiment is assumed to be the same as the encoder120 compliant with the H.263 coding system in the part not particularlydescribed.

[0117] The configuration of the encoder 20 differs from that of theconventional encoder 120 in including a hybrid predicting section 24 anda hybrid prediction decoding section 31 instead of the conventionalsubtracting section 124 and the conventional adding section 131,respectively.

[0118] Moreover, in the encoder 20 according to this embodiment, the“macro block mode” is determined by a hybrid predicting section 64instead of a motion estimating section 62.

[0119] The hybrid predicting section 24 divides each “macro block (firstimage block)” sent by the inputting section 21 into “sub-blocks (secondimage blocks)” of 8×8 pixels.

[0120] Moreover, the hybrid predicting section 24 calculates a “valueindicating predictive residual signal power” for each “sub-block”. Forexample, the hybrid predicting section 24 calculates a “SAD (Sums ofAbsolute Differences) between a sub-block within the macro block sent bythe inputting section 21 and a sub-block within the predicted imageblock sent by the motion compensating section 23, as the “valueindicating predictive residual signal power”.

[0121] Furthermore, the hybrid predicting section 24 compares thecalculated “SAD” with a predetermined threshold. When the “SAD” issmaller than the threshold, the hybrid predicting section 24 selects toapply an “inter-frame prediction mode (first prediction mode)” to thesub-block of interest. When the “SAD” is larger than the threshold, thehybrid predicting section 24 selects to apply au “intra-frame predictionmode (second prediction mode)” to the sub-block of interest.

[0122] In addition, after finishing the selection of the “predictionmode” for every sub-block within a macro block between application ofthe “inter-frame prediction mode (first prediction mode)” andapplication of the “intra-frame prediction mode (second predictionmode)”, the hybrid predicting section 24 examines the selection statesof the “prediction mode” in the entire macro block, so as to determinethe “macro block mode” of the macro block of interest.

[0123] Herein, the “macro block mode” includes “INTRA mode”, “INTERmode”, “INTER 4V mode”, and “INTRA/INTER mode”. Note that the “Macroblock mode” includes some other modes due to the differences in thepresence of a change in quantization parameters and the like.

[0124] In this embodiment, a description will be given of only a casewhere the “macro block mode” is selected based on the difference in theselection state of the “prediction mode” of each sub-block, but the“macro block mode” can be also selected according to the presence of achange in quantization parameters and the like,

[0125] Specifically, the hybrid predicting section 24 selects the “INTRAmode” as the “macro block mode”, when the “intra-frame prediction mode”is selected for every sub-block within the macro block.

[0126] When the “inter-frame prediction mode” is selected for everysub-block within the macro block, the hybrid predicting section 24selects the “INTER mode” or “INTER 4V mode” as the macro block mode”.

[0127] Moreover, when both the “intra-frame prediction mode” and the“inter-frame prediction mode” are selected together within the macroblock the hybrid predicting section 24 selects the “INTRA/INTER mode” asthe “macro block mode”.

[0128] In addition, the hybrid predicting section 24 sends the “marcoblock mode” determined for each macro block, to a variable lengthencoding section 27 and the hybrid prediction decoding section 31.

[0129] Furthermore, when the “INTRA/INTER mode” is selected as the“macro block mode”, the hybrid predicting section 24 sends the selectionstate of the “prediction mode” of each sub-block and the “motion vector”of each sub-block to which the “inter-frame prediction mode” is applied,to the variable length encoding section 27 and the hybrid predictiondecoding section 31.

[0130] Herein, when the “INTRA/INTER mode” is selected, the hybridpredicting section 24 may send one motion vector for each macro block tothe variable length encoding section 27 and the hybrid predictiondecoding section 31, in order to suppress the increase in the amount ofinformation necessary for coding the motion vectors.

[0131] Moreover, the hybrid predicting section 24 may directly send themotion vector for the “INTER mode” which is estimated by the motionestimating section 22, to the variable length encoding section 27 andthe hybrid prediction decoding section 31. As a result, the hybridpredicting section 24 does not have to calculate the motion vectoragain, and the increase in the amount of processing can be suppressed inthe hybrid predicting section 24.

[0132] Furthermore, the hybrid predicting section 24 may send a motionvector that most frequently appears among four motion vectors for the“INTER 4V mode” which are estimated by the motion estimating section 22,to the variable length encoding section 27 and the hybrid predictiondecoding section 31, as the motion vector for the macro block. As aresult, the variable length encoding section 27 can select a motionvector suitable for sub-blocks in which coding can be efficientlyperformed with the “inter-frame prediction mode” within the macro block,and increase the coding efficiency in the “INTRA/INTER mode”.

[0133] In addition, the hybrid predicting section 24 may search for andobtain a new motion vector when the “INTRA/INTER mode” is selected.

[0134] In such a case, the hybrid predicting section 24 searches for amotion vector which minimizes the SAD in combination with the“intra-frame prediction mode”, so as to achieve more efficient coding.

[0135] In other words, the hybrid predicting section 24 can implementthe motion vector search using the robust estimation. In the robustestimation, the hybrid predicting section 24 repeats obtaining the SADin each sub-block within the predicted image block in the macro block ofthe “INTER mode” and again searching for the motion vector afterremoving a sub-block with the largest SAD. Accordingly, it is possibleto search for a motion vector closer to the optimum.

[0136] As described above, the hybrid predicting section 24 constitutesprediction mode selection information generating moans for generating“prediction mode selection states (prediction mode selectioninformation)” indicating that the “inter-frame prediction mode (firstprediction mode) for “reducing temporal redundancy is applied to each ofsub-blocks (second image blocks) or that the “intra-frame predictionmode (second prediction mode) for reducing spatial redundancy is appliedto each of the sub-blocks (second image blocks), Herein, the sub-blocksare obtained by dividing a macro block (first image block).

[0137] Moreover, the hybrid predicting section 24 constitutes.predictive residual signal generating means for generating thepredictive residual signal by applying the selected “inter-frameprediction mode (first prediction mode)” or the selected “intra-frameprediction mode (second prediction mode)” to the sub-blocks (secondimage blocks).

[0138] The variable length encoding section 27 encodes the “quantizedorthogonal transformation coefficient” sent by the quantizing section 26as the variable length codes, and encodes predetermined information, forexample, information indicating the aforementioned prediction modeselection states a motion vector, and the like, as the variable lengthcodes based on the “macro block mode” sent by the hybrid predictingsection 24. The variable length encoding section 27 sends the encodedresult to the outputting section 28.

[0139] Herein, to express the “INTRA/INTER mode”, the definitions of themacro block modes of P pictures in the H.263 coding system are modifiedas shown in FIG. 8(a). In FIG. 8(a) the macro block modes with “MBtype=6, 7” are extended to express the “INTRA/INTER mode”.

[0140] The variable length encoding section 27 can determine which pieceof information (“COD”, “MCBPC”, “CBPY”, “DQUANT”, “MVD”, or “MVD2-4”)should be multiplexed with the bit stream 3 together with the “quantizedorthogonal transformation coefficients”, with reference to thedefinitions of the macro block modes.

[0141] In FIG. 6(a), the “COD” is a flag indicating whether or not amacro block of interest is “skip”. The “skip” indicates that the macroblock is reproduced by duplicating the image in the same spatialposition within the reference frame stored in the frame memory 32 and norequirement of additional particular information. In the case of “MBtype=Not coded”, the COD is 1, and the variable length encoding section27 does not multiplex information other than the “COD” for the macroblock of interest.

[0142] The “MCBPC” is a syntax element to express additionally “codedblock pattern information (CBP)” of a color-difference signal in themacro block of interest. The “coded block pattern information: CBP”means a flag indicating whether or not significant DCT coefficients tobe transmitted are present in sub-blocks which are the units for DCTtransformation.

[0143] The “CBPY” is a syntax element to express coded block patterninformation of a luminance signal in the macro block mode of interest.

[0144] The “DQUANT” indicates a modified value of the quantizationparameters (for example, a quantization step) of the macro block ofinterest..

[0145] The “MVD” is at least one motion vector. The “MVD2-4” indicatesthe second to fourth motion vectors transmitted in the case of “INTER 4Vmode”.

[0146] In the “INTRA/INTER mode”, to switch the “intra-frame predictionmode” and the “inter-frame prediction mode” for each sub-block forencoding, the “COD” is set to 0, and the “MCBPC” and “CBPY” aredetermined according to the distribution of the DCT coefficients.Moreover, the syntax (MB type=6, 7) for transmitting the “MVD” is usedfor sub-blocks to which the “inter-frame prediction mode” is applied.

[0147] Moreover, the syntax of the “block layer” in the H.263 codingsystem is extended as shown in FIG. 8(b) to show the selection state ofthe “intra-frame prediction mode” or the “inter-frame prediction mode”in each sub-block.

[0148] In FIG. 8(b), the “BTYPE” is composed of 1 bit and exists only inthe case of “MB type=6, 7”. Herein, the “BTYPE=0” indicates that thesub-block is encoded with the intra-frame prediction mode, and the“BTYPE=1” indicates that the sub-block is encoded with the inter-frameprediction mode.

[0149] The “INTRADC” is present only in the case that the “MB type=6, 7and the “BTYPE”=0.

[0150] As previously described, the variable length encoding section 27and the outputting section 28 constitute transmitting means for encodingand transmitting the prediction mode selection state (prediction modsselection information) and the predictive residual signal of eachsub-block (second image block).

[0151] The hybrid prediction decoding section 31 generates a “macroblock” constituting the “reference frame” used in the subsequent codingbased on the “macro block mode” sent by the hybrid predicting section24, and sends the generated macro block to the frame memory 32.

[0152] Specifically, when the “INTRA mode” is selected as the “macroblock mode”, the hybrid prediction decoding section 31 sets the“predictive residual signal” sent by an inverse orthogonal transformingsection 30 as a “macro block” constituting the “reference frame”, sincethe macro block of interest is constructed by the “intra-frameprediction”.

[0153] When the “INTER mode” or “INTER 4V mode” is selected as the“macro block mode”, the hybrid prediction decoding section 31 setsresult of adding up the “predictive residual signal” sent by the inverseorthogonal transforming section 30 and the “predicted image block” sentby the motion compensating section 23 as a “macro block” constitutingthe “reference frame”, since the macro block of, interest is constructedby the “inter-frame prediction”.

[0154] When the “INTRA/INTER mode” is selected as the “macro blockmode”, the hybrid prediction decoding section 31 switches the predictionmode to be applied to each sub-block, between the “intra-frameprediction mode” and “inter-frame prediction mode”.

[0155] When the “intra-frame prediction mode” is applied to thesub-block, the hybrid prediction decoding section 31 sets the predictiveresidual signal (sub-block basis)” sent by the inverse orthogonaltransforming section 30 as the “sub-block” constituting the “referenceframe”.

[0156] On the contrary, when the “inter-frame prediction mode” isapplied to the sub-block, the hybrid prediction decoding section 31 setsthe result of adding up the “predictive residual signal (sub-blockbasis)” sent by the inverse orthogonal transforming section 30 and the“predicted image block (sub-block basis)” sent by the motioncompensating section 23 as the “sub-block” constituting the “referenceframe”.

[0157] The decoder 40 according to this embodiment is shown in FIG. 7.Hereinafter, a description will be mainly given of a difference betweenthe decoder 40 according to this embodiment and the conventional decoder140 compliant with the H.263 coding system. The decoder 40 according tothis embodiment is assumed to be the same as the decoder 140 compliantwith the H.263 coding system in the part not particularly described.

[0158] The configuration of the decoder 40 is different from that of theconventional decoder 140 in including a hybrid prediction decodingsection 46 instead of the conventional adding section 146.

[0159] A variable length decoding section 42 decodes necessaryinformation (“quantized orthogonal transformation coefficients”, “macroblock modes”, “motion vectors”, and the like) for each macro block,starting from the head of each frame image in the compressed bit stream3 sent by an inputting section 41.

[0160] The variable length decoding section 42 sends the decoded“quantized orthogonal transformation coefficients”, “macro block modes”,and “motion vector” to a dequantizing section 43, the hybrid predictiondecoding section 46, and the motion compensating section 45,respectively.

[0161] Moreover, in the case where the “INTRA/INTER mode” is selected,the variable length decoding section 42 sends the “prediction modeselection states” and “motion vector” of each sub-block, to the hybridprediction decoding section 46.

[0162] As described above, the variable length decoding section 42constitutes prediction mode selection information decoding means ordecoding “prediction mode selection states (prediction mode selectioninformation)” which indicates that the “inter-frame prediction mode(first prediction mode) is applied to each of the sub-blocks (secondimage blocks) or that the “intra-frame prediction mode (secondprediction mode)” is applied to each of the sub-blocks (second imageblocks). The second image blocks are obtained by dividing a first imageblock, and the first blocks are obtained by dividing the video.

[0163] In the case where the “INTRA mode” is selected as the “Macroblock mode”, the hybrid prediction decoding section 46 sets the“predictive residual signal” sent by the inverse orthogonal transformingsection 44 as a “macro block” constituting the output video signal 4,since the macro block of interest is constructed with intra-frameprediction.

[0164] Moreover, in the case where the “INTER mode” or “INTER 4V mode”is selected as the “macro block mode”, the hybrid prediction decodingsection 46 sets the result of adding up the “predictive residual signal”sent by the inverse orthogonal transforming section 44 and the“predicted image block” sent by the motion compensating section 23 as a“macro block” constituting the output video signal 4 since the macroblock of interest is constructed with the motion compensated inter-frameprediction.

[0165] Furthermore, in the case where the “INTRA/INTER mode” is selectedas the “macro block mode”, the hybrid predicting section 46 switches theprediction mode to be used in each sub-block between the “intra-frameprediction mode” and the “inter-frame prediction mode”, based on the“prediction mode selection states” sent by the variable length decodingsection 42.

[0166] Herein, in the case where the “intra-frame prediction mode” isapplied to the sub-block, the hybrid prediction decoding section 46 setsthe “predictive residual signal (sub-block basis)” sent by the inverseorthogonal transforming section 44 as the “sub-block” constituting theoutput video signal 4.

[0167] On the contrary, in the case where the “inter-frame predictionmode” is applied to the sub-block, the hybrid prediction decodingsection 46 sets the result of adding up the “predictive residual signal(sub-block basis)” sent by the inverse orthogonal transforming section44 and the “predicted image block (sub-block basis)” sent by the motioncompensating section 23 as the “sub-block” constituting the output videosignal 4.

[0168] Furthermore, the, hybrid prediction decoding section 46 sends the“macro block” constituted of the sub-blocks obtained as described aboveto the outputting section 48, and sends the same “macro block” as a“macro block” constituting the “reference frame” used in the subsequentcoding to the frame memory 47.

[0169] As described above, the hybrid prediction decoding section 46constitutes video decoding means for decoding each of the second imageblocks of the output video signal (video) 4, based on the “inter-frameprediction mode (first prediction mode)” or “intra-frame prediction mode(second prediction mode)” selected according to the prediction modeselection states (prediction mode selection information).

OPERATION OF VIDEO CODING SYSTEM ACCORDING TO EMBODIMENT 1

[0170] The operation of the video coding system having theaforementioned configuration will be described with reference to FIG. 9.FIG. 9 is a flowchart showing processing in the hybrid predictingsection 24 of the encoder 20. Hereinafter, a description will be givenof the operation of the video coding systems focusing on the operationof the hybrid predicting section 24 of the encoder 20.

[0171] As shown in FIG. 9, the hybrid predicting section 24 divides amacro block sent by the inputting section 21 into sub-blocks of 8×8pixels in step 401.

[0172] In step 402, the hybrid predicting section 24 calculates the“SAD” for each sub-block between the sub-block within the macro blocksent by the inputting section 21 and a sub-block within the predictedimage block sent by the motion compensating section 23.

[0173] In step 403, the hybrid predicting section 24 compares thecalculated SAD with a predetermined threshold.

[0174] When the SAD is larger than the threshold, the hybrid predictingsection 24 selects to apply the “intra-frame prediction mode” to thesub-block of interest in step 404.

[0175] On the contrary, when the SAD is smaller than the threshold, thehybrid predicting section 24 selects to apply the “inter-frameprediction mode” to the sub-block of interest in step 405.

[0176] In step 406, the hybrid predicting section 24 judges whether ornot the aforementioned selection of the prediction mode is performed forevery sub-block within the macro block of interest. When the selectionis not performed for every sub-block, the hybrid predicting section 24returns to the step 402 and performs the aforementioned selection of theprediction mode for the rest of the sub-blocks.

[0177] When the aforementioned selection is performed for everysub-block, the hybrid predicting section 24 examines the prediction modeselection states in the entire macro block in step 407, and judgeswhether or not the “intra-frame prediction mode” is selected for everysub-block.

[0178] When the judgment is positive (YES), namely, when the“intra-frame prediction mode” is selected for every sub-block, thehybrid predicting section 24 vets the “macro block mode” of the macroblock of interest to the “INTRA mode” in step 408.

[0179] On the contrary, when the judgment is negative (NO), namely, whenthe “intra-frame prediction model is not selected for every sub-block,in step 409, the hybrid predicting section 24 examines the predictionmode selection states of the entire macro block, and judges Whether ornot the “inter-frame prediction mode” is selected for every sub-block.

[0180] When the judgment is positive (YES), namely, when the“inter-frame prediction mode” is selected for every sub-block, thehybrid predicting section 24 sets the “macro block mode” of the macroblock of interest to the “INTER mode” or “INTER 4V mode” in step 410.

[0181] Herein, which is selected between the “INTER mode” and “INTER 4Vmode” is determined based on the presence of difference in motionvectors within the macro block.

[0182] On the contrary, when the judgment is negative (NO), namely, whenthe “inter-frame prediction mode” is not selected for every sub-block,the hybrid predicting section 24 sets the “macro block mode” of themacro block of interest to the “INTRA/INTER mode” in step 411.

[0183] In step 412, the hybrid predicting section 24 sends the “macroblock mode” determined for each macro block, to the variable lengthdecoding section 27 and the hybrid prediction decoding section 31.

[0184] However, when the “INTRA/INTER mode” is set as the “macro blockmode”, the hybrid predicting section 24 sends the “macro block mode”,the “prediction mode selection state” of each sub-block, and the “motionvector” of each sub-block (sub-block to which the inter-frame predictionmode to applied), to the variable length encoding section 27 and theprediction decoding section 31.

[0185] The description has been given of the video coding transmissionsystem according to this embodiment assuming to have a configurationcompliant with ITU-T H.263. However, the present invention is notlimited to this and applicable to various video coding transmissionsystems which perform coding in units equivalent to the macro blocks(first image blocks) by using prediction modes of the “inter-frameprediction mode (first prediction mode)” and “intra-frame predictionmode (second prediction mode)”.

[0186] Moreover, the “INTRA/INTER mode” is not limited to theconfiguration shown in this embodiment and can employ various syntaxesindicating which of the “inter-frame prediction mode” or the“intra-frame prediction mode” is applied to each sub-block (second imageblock) within the macro block (first image block).

[0187] Similarly, the “BTYPE” is not limited to the configuration shownin this embodiment and can employ various syntaxes indicating which ofthe “inter-frame prediction mode” or the “intra-frames prediction mode”is applied to each sub-block.

[0188] Furthermore, in this embodiment, the hybrid predicting section 24compares the calculated SAD with a predetermined threshold and shallselect the “macro block mode”, but the selection criterion is notlimited to the configuration shown in this embodiment.

[0189] It is possible to employ various selection criteria for selectingthe “macro block mode” which enables more efficient coding. For example,the compressed bit stream 3 for the macro block of interest is actuallyconstructed using each “macro block mode”, and the “macro block mode”constructing the bit stream 3 having smaller amount of bits may beselected.

FUNCTION AND EFFECT OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 1

[0190] With the video coding transmission system according to Embodiment1, the hybrid predicting section 24 can optimally generate thepredictive residual signal in which the spatial redundancy or thetemporal redundancy is reduced from sub-blocks. Accordingly, a codingprocess can be efficiently performed even when the portion which isdesirable to be encoded with the inter-frame prediction mode and theportion which is desirable to be encoded with the intra-frame predictionmode are mixed in a macro block.

CONFIGURATION OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 2

[0191] A description will be given of a video coding transmission systemaccording to Embodiment 2 with reference to FIGS. 10 and 11. FIG. 10 isa block diagram showing an encoder 60 used in the video codingtransmission system according to this embodiment. FIG. 11 is a blockdiagram showing a decoder 60 used in the video coding transmissionsystem according to this embodiment.

[0192] The video coding transmission system according to this embodimentis compliant with the ITU-T H.26L coding system and has a configurationto which necessary modifications are added on that basis. The H.26Lcoding system is described in the ITU-T SG16 Q.6 VCEG-M81, “H.26 TestModel Long Term Number7 (TML-7) draft0,” and the like.

[0193] Compared to the video coding transmission system according to theaforementioned Embodiment 1, the video coding transmission systemaccording to this embodiment is modified in terms of miniaturization ofthe unit of “sub-blocks”, a variety of motion vector estimation methods,and an adoption of a loop filter 73 and a spatial predicting section 74.

[0194] First, a description will be given of a difference related to theaforementioned miniaturization of the unit of “sub-blocks” in theencoder 60. In this embodiment, the unit of “sub-blocks” to be subjectedto orthogonal transformation is “4×4 pixels” which is smaller than theunit of “8×8 pixels” in the case of the encoder 20 according to theaforementioned embodiment. Therefore, the minimum unit for predictionmode selection and the minimum unit for motion vector estimation is asub-block of “4×4 pixels”.

[0195] Second, a description will be given of a difference related tothe aforementioned diversification of motion vector estimation methodsin the encoder 60.

[0196] The motion estimating section 62 estimates “motion vectors ofeach sub-block of 4×4 pixels as the minimum unit.

[0197] Specifically, the motion estimating section 62 estimates motionvectors related to all seven types of vector estimation units (16×16,16×8, 8×16, 8×8, 4×8, 8×4, and 4×4) shown in FIG. 12 of each sub-blockreceived through an inputting section 61 by using the reference framestored in a frame memory 72.

[0198] Moreover, the motion estimating section 62 sends the estimatedmotion vectors and the motion estimation units used for estimating therespective motion vectors of interest, to a motion compensating section63.

[0199] The motion compensating section 63 generates a “predicted imageblock” using each of the received motion vectors and a reference framestored in the frame memory 72. The motion compensating section 63 sendsthe generated predicted image block and the motion estimation unitcorresponding to the predicted image block of interest, to the hybridpredicting section 64.

[0200] Third, a description will be given of a difference related to theaforementioned adoption of the spatial predicting section 74 in theencoder 60.

[0201] As shown in FIG. 13, the spatial predicting section 74 performsintra-frame pixel value prediction by several types of pixel valueprediction methods which use pixel values of sub-blocks adjacent to thesub-block of interest, and generates the predicted image blocks, foreach sub-block (4×4 pixels) within the macro block sent by the inputtingsection 61.

[0202] Herein, the spatial predicting section 74 requires sub-blockswhich have been subjected to the series of processing including theorthogonal transformation, quantization, dequantization, and inverseorthogonal transformation, as the aforementioned adjacent sub-blocks,when performing the aforementioned intra-frame pixel value prediction.Therefore, in this embodiment, the series of processing including theorthogonal transformation, quantization, dequantization, and inverseorthogonal transformation is performed for each of sub-blocks (4×4pixels).

[0203] Moreover, the spatial predicting section 74 performs theaforementioned intra-frame pixel value prediction by using sub-blocks(4×4 pixels) which have been subjected to the aforementioned series ofprocessing.

[0204] The spatial predicting section 74 sends the generated predictedimage block and the pixel value prediction method corresponding to thepredicted image block of interest, to the hybrid predicting section 64and the hybrid prediction decoding section 71.

[0205] The hybrid predicting section 64 selects a motion estimation unitcorresponding to the “predicted image blocks (sub-block basis)” whichresults in the smallest SAD between the sub-block within the macro blocksent by the inputting section 61 and “predicted image blocks (sub-blockbasis)” generated by the motion compensating section 63. Herein, thehybrid predicting section 64 applies the “inter-frame prediction mode”corresponding to the aforementioned motion estimation unit, to thesub-block of interest.

[0206] Moreover, The hybrid predicting section 64 selects a pixel valueprediction method corresponding to the “predicted image blocks(sub-block basis)” which results in the smallest SAD between thesub-block within the macro block sent by the inputting section 61 and“predicted image blocks (sub-block basis)” generated by the spatialpredicting section 74. The hybrid predicting section 64 applies the“intra-frame prediction mode” corresponding to the aforementioned pixelvalue prediction method, to the sub-block of interest.

[0207] Herein, the hybrid predicting section 64 compares the first SADin the case of applying the “inter-frame prediction mode” with thesecond SAD in the case of applying the intra-frame pixel valueprediction mode”, and applies the prediction mods with the smaller SADto the sub-block of interest.

[0208] The hybrid predicting section 64 applies the “prediction mode” asdescribed above to every sub-block within the macro block, for example,sequentially from the upper left block to the lower right block.

[0209] Note that, the “macro block mode” in the case where the hybridpredicting section 64 applies the “inter-frame prediction mode”corresponding to the aforementioned motion estimation unit or the“intra-frame prediction mode” corresponding to the above pixel valueprediction method to every sub-block within the macro block is the“INTRA/INTER mode”.

[0210] Moreover, the hybrid predicting section 64 compares the SADs(SADs calculated using the macro block sent by the inputting section 61)in the cases where the “INTRA/INTER mode”, “INTER mode”, and “INTRAmode” are applied to the macro block, and sends the “macro block mode”with the smallest SAD to a variable length encoding section 67 as the“macro block mode” of the macro block of interest.

[0211] As a result, the hybrid predicting section 64 can apply the“inter-frame prediction mode” or “intra-frame prediction mode” to eachsub-block, and search for a combination thereof achieving the mostefficient coding.

[0212] When the “INTRA/INTER mode” is selected as the “macro blockmode”, the hybrid predicting section 64 sends the “macro block mode”,the “prediction mode selection state” of each sub-block, and the “motionvector” or “pixel value prediction method” for each sub-block to thevariable length encoding section 67 and the hybrid prediction decodingsection 71.

[0213] When the “INTRA/INTER mode” is selected as the “macro block mode”and the “intra-frame pixel value prediction mode” is selected for thesub-block of interest, the hybrid prediction decoding section 74 adds upthe predicted image block sent by the spatial predicting section 74 andthe predictive residual signal sent by the inverse orthogonaltransforming section 70, and sends the added result to the loop filter73.

[0214] Moreover, when the “INTRA/INTER mode” is selected as the “macroblock mode” and the “inter-frame prediction mode” is selected for thesub-block of interest, the hybrid prediction decoding section 71 adds upthe predicted image block sent by the motion compensating section 63 andthe predictive residual signal sent by the inverse orthogonaltransforming section 70, and sends the added result to the frame memory72 through the loop filter 73.

[0215] Fourth, a description will be given of a difference related tothe aforementioned adoption of the loop filter 73 in the encoder 60.

[0216] The loop filter 73 serves to reduce deterioration such asdistortion of sub-blocks, by performing filtering in sub-blocks for theresult sent by the hybrid prediction decoding section 71.

[0217] In addition, in the video coding transmission system according tothis embodiment, the definitions of the macro block modes for the Ppictures in the H.26L coding system are modified as shown in FIG. 14(a)to indicate the “macro block mode” which is the “INTRA/INTER mode”.

[0218] In FIG. 14(a), the “Intra_pred_mode” indicates the pixel valueprediction method used for a sub-block to which the “intra-frame pixelvalue prediction mode” is applied.

[0219] The “Ref_frame” indicates the motion estimation unit used for asub-block to which the “inter-frame prediction mode” is applied.

[0220] Moreover, the “Intra_pred_mode” is extended as shown in FIG.14(b) to describe the “prediction mode selection state” of eachsub-block. The portion of the “INTER_PRED: 6” is the extension.

[0221] In this embodiment, as shown in FIG. 14(a), the macro block modecorresponding to the “INTRA 4×4 mode” defined in the conventional H.26Lcoding system is changed to the “INTRA/INTER mode”.

[0222] In the conventional H.26L coding system, when the “INTRA 4×4mode” is selected as the macro block mode, the MB type is assigned to 7.

[0223] On the other hand, as shown in FIG. 14(b), the “INTRA/INTER mode”according to this embodiment is defined in such a manner that the“INTER_PRED” is added to the Intra_pred_mode” in the macro block modewith “MB type=7” assigned, so as to select not only the intra-framepixel value prediction mode but also the inter-frame prediction mode foreach sub-block of 4×4 pixels.

[0224] The definition of the “INTRA/INTER mode” is not limited to theabove method and may be made separately from the “INTRA 4×4 mode”. Forexample, the “INTRA/INTER mode” may be defined by assigning differentmacro block modes according to the prediction mode selection states ofeach of sub-blocks.

[0225] As a result, the “Intra_pred_mode” allows identifying the numberof sub-blocks to which the intra-frame prediction mode is applied withinthe macro block and the number of sub-blocks to which the inter-framesprediction mode is applied within the macro block.

[0226] Moreover, the encoder 68 sends the “Intra_pred_mode” before theinformation concerning the inter-frame prediction mode such as the “MV”and the “Ref_frame”, in a macro block to which the “INTRA/INTER mode” isapplied. Accordingly, the encoder 60 can send the syntax for the“INTRA/INTER mode” without waste in the case where such informationconcerning the inter-frame prediction is not required.

[0227] Furthermore, when the “INTRA/INTER mode” is selected, the hybridpredicting section 64 may send a motion vector per one macro block tothe variable length encoding section 67 and the hybrid predictingdecoding section 71, in order to suppress the increase in the amount ofinformation necessary for coding motion vectors.

[0228] The hybrid predicting section 64 may obtain a prediction valuefrom a motion vector of a macro block (INTER mode) already coded amongmacro blocks adjacent to the current macro block, and set the differencebetween the prediction value of interest and the current macro block asa motion vector of the current macro block.

[0229] In addition, the hybrid predicting section 64 may directly sendthe motion vector for the “INTER 16×16 mode” which to previouslyestimated by the motion estimating section 62, to the variable lengthencoding section 67 and the hybrid prediction decoding section 71.

[0230] The hybrid predicting section 24 may send a motion vector thatthe most frequently appears among the motion vectors of each sub-blockestimated by the motion estimating section 62, to the variable lengthencoding section 67 and the hybrid prediction decoding section 71, as amotion vector of the current macro block.

[0231] As a result, it ;a possible to select a motion vector suitablefor sub-blocks which can be efficiently coded with the “inter-frameprediction mode” within the macro block, and increase the codingefficiency in the “INTRA/INTER mode”.

[0232] The decoder 80 according to this embodiment is shown in FIG. 11.Hereinafter, a description will be mainly given of differences betweenthe decoder 40 according to the aforementioned Embodiment 1 and thedecoder 80 according to this embodiment. The decoder 80 according tothis embodiment is the same as the decoder 40 according to theaforementioned Embodiment 1 in the part not particularly described.

[0233] The configuration of the decoder 80 according to this embodimentdiffers from that of the decoder 40 according to the aforementionedEmbodiment 1 in including a loop filter 89 and a spatial predictingsection 90.

[0234] The loop filter 89 includes the same function as the loop filter73 of the encoder 60. The loop filter 89 serves to reduce deteriorationsuch as distortion of sub-blocks, by performing filtering in sub-blocksto the result sent by a hybrid prediction decoding section 86.

[0235] The spatial predicting section 90 performs the intra-frame pixelvalue prediction by the pixel value prediction method sent by thevariable length decoding section 82, for each sub-block (4×4 pixels)within an image of the reference frame stored in the frame memory 72, soas to generate a predicted image block.

[0236] Specifically, when the “INTER mode” is selected as the “macroblock mode”, the hybrid prediction decoding section 86 sets the resultof adding up the “predictive residual signal” sent by the inverseorthogonal transforming section 84 and the “predicted image block” sentby the motion compensating section 85 as a “macro block” constitutingthe output video signal 4, since the macro block of interest isconstructed by the motion compensated inter-frame prediction.

[0237] Moreover, when the “INTRA mode” is selected as the “macro block”,the hybrid prediction decoding section 86 sets the result of adding upthe “predictive residual signal” sent by the inverse orthogonaltransforming section 84 and the “predicted image block” sent by thespatial predicting section 90 as a macro block constituting the outputvideo signal 4, since the macro block of interest is constructed by theintra-frame prediction.

[0238] Furthermore, when the “INTRA/INTER mode” is selected as the“macro block mode”, the hybrid prediction decoding section 86 switchesthe prediction mode to be used for each sub-block of 4×4 pixels betweenthe “intra-frame prediction mode” and the “inter-frame prediction mode”.

[0239] In other words, when the “intra-frame pixel value predictionmode” is applied to the sub-block of interest, the hybrid predictiondecoding section 86 sets the “predictive residual signal (sub-blockbasis)” sent by the inverse orthogonal transforming section 44 as the“sub-block” constituting the output video signal 4.

[0240] On the contrary, when the “intra-frame prediction mode” isapplied to the sub-block of interest, the hybrid prediction decodingsection 86 sets the result of adding up the “predictive residual signal(sub-block basis)” sent by the inverse orthogonal transforming section44 and the “predicted image block” sent by the spatial predictingsection 90 as the “sub-block” constituting the output video signal 4.

[0241] The hybrid prediction decoding section 86 generates a “macroblock” constituting the output video signal 4 from these “sub-blocks”.

[0242] The variable length decoding section 82 Constitutes predictionmode related information decoding means for decoding prediction moderelated information (motion vectors or pixel value prediction methods)necessary for performing the first or second prediction mode.

OPERATION OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TO EMBODIMENT 2

[0243] The operation of the video coding transmission system with theaforementioned configuration will be described with reference to FIG.15. FIG. 15 is a flowchart showing processing in the hybrid predictingsection 64 of the encoder 60. Hereinafter, a description will be givenof the operation of the video coding transmission system, focusing onthe operation of the hybrid predicting section 64 of the encoder 60.

[0244] As shown in FIG. 13, the hybrid predicting section 64 divides amacro block sent by the inputting section 61 into sub-blocks of 4×4pixels in step 901.

[0245] In step 902, the hybrid predicting section 64 receives from themotion compensating section 63 a plurality of predicted image blocks(sub-block basis) which have been generated by the motion compensatedinter-frame prediction using a plurality of motion estimation units.Moreover, the hybrid predicting section 64 receives from the spatialpredicting section 74 a plurality of predicted image blocks (sub-blockbasis) which have been generated by the intra-frame pixel valueprediction using a plurality of pixels value prediction methods.

[0246] In step 903, the hybrid predicting section 64 selects a predictedimage block that results in the smallest first SAD between a sub-blockwithin the macro block sent by the inputting section 61 and one out ofthe plurality of predicted image blocks (sub-block basis) from themotion compensating section 63.

[0247] Moreover, the hybrid predicting section 64 selects a predictedimage block that results in the smallest second BAD between a sub-blockwithin the macro block sent by the inputting section 61 and one out ofthe plurality of predicted image blocks (sub-block basis) from thespatial predicting section 74.

[0248] In step 904, the hybrid predicting section 64 compares theaforementioned first 8AD with the aforementioned second SAD for eachsub-block. When the first SAD is smaller, the hybrid predicting section64 sets the prediction mode of the sub-block of interest to the“inter-frame prediction mode” in step 905. When the second SAD issmaller, the hybrid predicting section 64 sets the prediction mode ofthe sub-block of interest to the “intra-frame pixel value predictionmode” in step 906.

[0249] In step 907, the hybrid predicting section 64 judges whether ornot the above selection is performed for every sub-block within themacro block of interest. When the selection is not performed for everysub-block, the hybrid predicting section 64 returns to the step 902 andperforms the above selection of the prediction mode for the rest of thesub-blocks.

[0250] When the aforementioned selection of the prediction mode isperformed for every sub-block, in step 908, the hybrid predictingsection 64 compares the SAD for the “INTRA/INTER mode” which is the sumof the SADs for the entire macro block of interest, the SAD of thepredicted image block by the motion compensated inter-frame predictionwhich is sent by the motion compensating section 63, with the SAD of thepredicted image block by the intra-frame image pixel value predictionwhich is sent by the spatial predicting section 74.

[0251] The hybrid predicting section 64 then applies the prediction modewith the smallest SAD as the “macro block mode” to the macro block ofinterest.

[0252] In step 909, the hybrid predicting section 64 sends this “macroblock mode” to the variable length encoding section 67 and the hybridprediction decoding section 71.

[0253] Herein, when the “INTRA/INTER mode” is selected as the “macroblock mode”, the hybrid predicting section 64 sends the “macro blockmode”, the “prediction mode selection states” of individual sub-blocks,and the “motion vectors” or the “pixel value prediction methods”, to thevariable length encoding section 67 and the hybrid prediction decodingsection 71.

[0254] The video coding transmission system according to this embodimenthas been described as a construction compliant with ITU-T H.263. Thepresent invention is not limited to this and applicable to various typesof video coding transmission systems which perform a coding processusing prediction modes of the “inter-frame prediction mode (firstprediction mode)” and “intra-frame prediction mode (second predictionmode)” in units equivalent to the macro blocks (first image blocks).

[0255] Moreover, the “INTRA/INTER mode” is not limited to theconfiguration shown in this embodiment and can employ various syntaxesindicating which of the “inter-frame prediction mode” or “intra-frameprediction mode” is applied to each sub-block (second image block)within the macro block (first image block).

[0256] For example, the “INTRA/INTER mode” may be used separately fromthe “INTRA 4×4 mode”. In this case, when the prediction mode selectionstate of every sub-block within the macro block is the “intra-framepixel value prediction mode”, the “INTRA 4×4 mode” is applied to themacro block of interest. On the contrary, when the prediction modeselection states of the sub-blocks within the macro block include the“inter-frame prediction mode”, the “INTRA/INTER mode” is applied to themacro block of interest.

[0257] At this time, in the “INTRA 4×4 mode”, the “Intra_pred_mode”indicating the prediction mode selection states uses a table notextended, which is used in the conventional H.26L coding system. On theother hand, in the “INTRA/INTER mode”, the “Intra_pred_mode” indicatingthe prediction mode selection states uses a table extended as shown inFIG. 14 (b) or a table in which part of items concerning the“intra-frame prediction mode” other than the “INTER_PRED” is omitted.Accordingly, the syntaxes for the “INTRA 4×4 mode” and the “INTRA/INTERmode” can be sent efficiently.

[0258] Similarly, the “Intra_pred_mode” is not limited to theconfiguration shown in this embodiment and can employ various syntaxesindicating which of the “inter-frame prediction mode” or “intra-frameprediction mode” is applied to each sub-block.

[0259] Furthermore, in this embodiment, the hybrid predicting section 64compares the calculated SAD with the predetermined threshold so as toselect the “macro block mode”. However, this selection criterion is notlimited to this configuration shown in this embodiment.

[0260] It is possible to employ various selection criteria for selectingthe “macro block mode” which enables more efficient coding. For example,the compressed bit stream 3 for the macro block of interest is actuallyconstructed with each “macro block mode”, and the “macro block mode”constructing the macro block having smaller amount of bits of the bitstream 3 may be selected.

FUNCTION AND EFFECT OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 2

[0261] With the video coding transmission system according to Embodiment2, the hybrid predicting section 64 can optimally generate a predictiveresidual signal in which the spatial redundancy or the temporalredundancy is reduced from sub-blocks. Accordingly, it is possible toperform efficient coding even when the portion which :s desirable to beencoded with the inter-frame prediction mode and the portion which isdesirable to be encoded with the intra-frame prediction mode are mixedin a macro block.

CONFIGURATION OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 3 TO THE PRESENT INVENTION

[0262] A description will be given of a video coding transmission systemaccording to Embodiment 3 of the present invention with reference to thedrawings. Similar to the case of the Embodiment 2 of the presentinvention, the video coding transmission system according to thisembodiment is compliant with the ITU-T H.26L coding system and providedwith a configuration to which modifications necessary for carrying outthis present invention are added on that basis. The configuration of thevideo coding transmission system according to this embodiment is shownin FIGS. 10 and 11 as well as the case of the Embodiment 2.

[0263] Compared to the video coding transmission system according to theEmbodiment 2, the video coding transmission system according to thisembodiment is modified in the following points.

[0264] First, the video coding transmission system according to thisembodiment is modified in terms of setting a “sub-block” that is a unitfor selecting which of the “inter-frame prediction mode” and“intra-frame prediction mode” is to be applied as the “prediction mode”to the “motion estimation unit” shown in FIG. 12, instead of the “blockunit of 4×4 pixels” that to a unit for orthogonal transformation.

[0265] Second, the video coding transmission system according to thisembodiment is modified in terms of defining the syntax necessary forselecting application of the “inter-frame prediction mode” orapplication of “intra-frame prediction mode” on such sub-block basis.

[0266] Hereinafter, a description will be given of only differencesbetween the encoder 60 according to the Embodiment 2 and the encoder 60according to this embodiment. The encoder 60 according to thisembodiment is assumed to be the same as the encoder 60 according to theaforementioned Embodiment 2 in the part not particularly described.

[0267] In the description of this embodiment, unlike in the case of theEmbodiment 2, a block of 4×4 pixels that is the unit of orthogonaltransformation is referred to as an “orthogonal transformation block”,instead of the “sub-block”, because of the aforementioned difference ofthe unit of “sub-blocks”.

[0268] In the encoder 60 according to this embodiment, the motionestimating section 62 estimates “motion vectors” based on seven types ofmotion estimation units (16×16, 16×8, 8×16, 8×8, 4×8, 8×4, and 4×4)shown in FIG. 12, namely, on a “sub-block” basis, and sends theestimated “motion vectors” as “motion vectors” in respective “predictionmode (specifically, corresponding to the respective motion estimationunits shown in FIG. 12)”.

[0269] The motion compensating section 63 applies the “motion vectors”sent by the motion estimating section 62 to the reference frame storedin the frame memory 72 so as to generate “predicted image blocks(sub-block basis)”. The motion compensating section 63 then sends thegenerated “predicted image block (sub-block basis)” to the hybridpredicting section 64.

[0270] As shown in FIG. 11, the spatial predicting section 74 performsseveral types of pixel value predicting methods which use pixel valuesof orthogonal, transformation blocks adjacent to the orthogonaltransformation block of interest for each “orthogonal transformationblock (4×4 pixels)” within the macro block sent by the inputting section61, so as to generate predicted image blocks (sub-block basis). Thespatial predicting section 74 sends the generated “predicted imageblocks (sub-block basis)” to the hybrid predicting section 64.

[0271] The spatial predicting section 74 performs the aforementionedgeneration of predicted image blocks for every orthogonal transformationblock included within one sub-block, and selects a pixel valueprediction method which minimizes the SAD from the sub-block receivedvia the inputting section 61.

[0272] Herein, the spatial predicting section 74 also performs theintra-frame pixel value prediction for the entire macro block (16×16) byseveral types of intra-frame pixel value prediction methods using pixelvalues of macro blocks adjacent to the macro block of interest, so as togenerate a predicted image block.

[0273] The spatial predicting section 74 may send, to the hybridpredicting section 64, an identifier indicating the pixel valueprediction method instead of the aforementioned predicted image blocks(sub-block basis) generated by using the selected pixel value predictionmethod.

[0274] The aforementioned orthogonal transformation blocks adjacent tothe orthogonal transformation block of interest are required to havebeen subjected to the series of processing including the orthogonaltransformation, quantization, dequantization, and inverse orthogonaltransformation. Accordingly, the spatial predicting section 74 performsthe aforementioned intra-frame pixel value prediction by usingorthogonal transformation blocks which have been subjected to thisseries of processing.

[0275] In such a case, when am adjacent orthogonal transformation blockis included in the same macro block, the prediction mode used for theadjacent orthogonal transformation block has not been fixed yet in somecases.

[0276] In this case, the aforementioned coding and decoding may beperformed using a prediction mode which is the candidate for theadjacent orthogonal transformation block of interest. Alternatively, inthe case where the adjacent orthogonal block is included in the samemacro block, the intra-frame pixel value prediction may be performed onthe assumption that the adjacent orthogonal transformation block ofinterest is outside the macro block.

[0277] The hybrid predicting section 64 divides the macro block sent bythe inputting section 61 into “sub-blocks (motion estimation units)”,and applies the “inter-frame prediction mode” or “intra-frame predictionmode” to each sub-block, so as to search for the combination thatprovides the most efficient coding.

[0278] Specifically, the hybrid predicting section 64 sequentiallyexamines the “prediction modes” corresponding to the seven types ofmotion estimation units shown in FIG. 12 as follows.

[0279] The hybrid predicting section 64 calculates the SAD, between the“predicted image block” and the sub-block within the macro block sent bythe inputting section 61, for each sub-block in each prediction mode.Herein, the “predicted image block” is the “predicted image block(sub-block basis)” by the motion compensated inter-frame prediction sentby the motion predicting section 63, or the “predicted image block(orthogonal transformation block basis)” by the intra-frame pixel valueprediction sent by the spatial predicting section 74. The hybridpredicting section 64 then sets, prediction method (motion compensatedinter-frame prediction or intra-frame pixel value prediction) with thesmaller SAD as the prediction method for the sub-block of interest.

[0280] The “predictive residual signal (sub-block basis)” generatedbased on the predicted image sub-block by this prediction method issubjected to the series of coding and encoding processing including theorthogonal transformation, quantization, dequantization, and inverseorthogonal transformation. The aforementioned processing is performedfor every sub-block (sequentially from the upper left sub-block to thelower right sub-block) within the macro block.

[0281] After performing the aforementioned processing for everysub-block within the macro block, the hybrid predicting section 64 holdsthe sum of the SADs for the entire macro block as the SAD in the “macroblock mode (INTER 16×16 mode, etc.)” corresponding to each motionestimation unit. The hybrid predicting section 64 then performs theaforementioned processing in the same manner for a “macro block mode(INTER 16×16 mode, etc.) corresponding to a motion estimation unit forwhich the examination has not been performed.

[0282] After performing the aforementioned processing for the “macroblock modes” corresponding to all the motion estimation units, thehybrid predicting section 64 compares the hold SADs for the respective“macro block modes”.

[0283] The hybrid predicting section 64 sets the “macro block mode” withthe smallest SAD as the “macro block-mode (INTER 16×16 mode, etc.)wusing the inter-frame prediction of the macro block of the interest, andholds the set SAD.

[0284] The hybrid predicting section 64 compares the first SAD in thecase of performing the motion compensated inter-frame prediction, withthe second SAD in the case of performing the intra-frame pixel valueprediction for the entire macro block of interest sent by the spatialpredicting section 74.

[0285] The hybrid predicting section 64 sets the macro block mode whichfinally provides the smallest SAD as the “macro block mode” of the macroblock of interest, and sends this “macro block mode” to the variablelength encoding section 67 and the hybrid prediction decoding section71.

[0286] When the “INTER mode” is selected as the “macro block mode”, thehybrid predicting section 64 sends the “macro block mode”, the“prediction mode selection state” of each sub-block, and the “motionvector” or “pixel prediction method”, to the variable length encodingsection 67 and the hybrid prediction decoding section 71.

[0287] The hybrid prediction decoding section 71 adds up the “predictedimage block (sub-block basis)” sent by the motion compensating section63 and “predictive residual signal (sub-block basis)”, and sends theresult to the loop filter 73, in the case where the “INTER mode” isselected as the “macro block mode” and the “inter-frame prediction mode”is selected for the sub-block of interest.

[0288] The hybrid prediction decoding section 71 adds up the “predictedimage block (sub-block basis)” sent by the spatial predicting section 74and the “predictive residual signal (sub-block basis)”, and sends theresult to the frame memory 72 through the loop filter 72, in the casewhere the “INTER mode” is selected as the “macro block mode” and the“intra-frame pixel value prediction mode” is selected for the sub-blockof interest.

[0289] The loop filter 73 serves to suppress the deterioration such asthe distortion of the sub-block, by performing filtering in sub-blocksfor the result sent by the hybrid prediction decoding section 71.

[0290] In the video coding transmission system according to thisembodiment, the syntax of the “INTER mode” is extended as follows toallow the “inter-frame prediction mode” and the “intra-frame pixel valueprediction mode” to be mixed in a macro block.

[0291] Specifically, in the video coding transmission system accordingto this embodiment, when the “INTER mode” is selected as the “macroblock mode”, the definitions of the “Ref_frame” and the syntax in macroblock modes of the P picture in the H.26L coding system are extended asshown in FIGS. 16(a) and 16(b) and FIG. 17, to show the selection statethat is the “inter-frame prediction mode” or “intra-frame pixel valueprediction mode”.

[0292]FIG. 16(a) shows a syntax in the macro block mode of P pictures inthe conventional H.26L. In FIG. 16(a), the “Intra_pred_mode” indicatesthe prediction method (motion compensated inter-frame prediction orintra-frame pixel value prediction) selected for each orthogonaltransformation block, in the case where the “INTRA 4×4 mode” is selectedin the conventional H.26L coding system.

[0293] In the H.26L coding system, the “Ref_frame” indicates whichreference frame is used for the macro block, since a “motion vector” canbe estimated using a plurality of reference frames in the motioncompensated inter-frame prediction.

[0294] The “MVD” indicates difference information of motion vectors. The“Texture Coding Syntax” indicates information such as the orthogonaltransformation coefficients other than the aforementioned information.

[0295] The syntax shown in FIG. 16(a) is extended as shown in FIG.16(b). The “Ref_frame” is transmitted as many times as the number ofsub-blocks which is discriminated by the “macro block mode” indicated bythe “MB_type”.

[0296] The definition of the “Ref_frame” extended therefor is shown inFIG. 17. In FIG. 17, the maximum number of reference frames is “5”. Thecode next to the code indicating the “5” is designed to indicate the“intra-frame prediction mode”.

[0297] In the H.26L coding system, the maximum number of referenceframes is variable and can take a value other than “5”. Herein, themaximum number of reference frames is previously known in the encoder 60and the decoder 80, and it is considered that similar settinginformation is transmitted in advance in actual applications.

[0298] Accordingly, if the code next to the code indicating the maximumnumber of reference frames is designed to be assigned to the codeindicating the “intra-frame prediction mode”, the extended “Ref_frame”can be uniquely defined in the encoder 60 and the decoder 80 despite themaximum number of reference frames.

[0299] Accordingly, the selection state of the “inter-frame predictionmode” or “intra-frame pixel value prediction mode” for each sub-block istransmitted by using the “Ref_frame”.

[0300] As a result, the “inter-frame prediction mode” and the“intra-frame pixel value prediction mode” are allowed to be mixed in themacro block in which the “INTER mode” is selected.

[0301] Moreover, the “Ref_frame” is designed to be transmitted beforethe “Intra_pred_mode”, since the selection state that is the“inter-frame prediction mode” or “intra-frame prediction mode” isdiscriminated by the “Ref_frame”. Moreover, it is designed to transmit“Intra_pred_mode” as many times as the number of sub-blocks of the“intra-frame prediction mode” indicated by the “Ref_frame”, in the casewhere the “Ref_frame” indicates the presence of sub-blocks of the“intra-frame prediction mode”.

[0302] Transmitting the “Ref_frame” before the “Intra_pred_mode” allowsto send the syntax of each mode with no waste, even in the case wherethere is no need for information concerning the inter-frame predictionmode such as the “MVD” or information concerning the intra-frame pixelvalue prediction mode such as the “Intra_pred_mode”.

[0303] In this case, the “Ref_frame” is transmitted for each sub-blockunlike the case of the conventional H.26L coding system.

[0304] Accordingly, it becomes possible to select the “inter-frameprediction mode” or “intra-frame pixel value prediction mode” for eachsub-block as well as to select a reference frame for each sub-block,which was selected for each macro block in the case of the conventionalH.26L coding system.

[0305] The configuration of the decoder 80 according to this embodimentis the same as that of the decoder 80 according to the Embodiment 2except using sub-blocks that are the motion estimation unit instead ofsub-blocks of 4×4 pixels

OPERATION OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TO EMBODIMENT 3

[0306] The operation of the video coding transmission system having theaforementioned configuration will be described with reference to FIG.18. FIG. 18 is a flowchart showing processing in the hybrid predictingsection 64 of the encoder 60 in this embodiment. Hereinafter, adescription will be given of the operation of the video codingtransmission system focusing on the operation of the hybrid predictingsection 64 of the encoder 60.

[0307] As shown in FIG. 15, the hybrid predicting section 64 divides themacro block sent by the inputting section 61 into sub-blocks(corresponding to the motion estimation unit) in step 1501.

[0308] In step 1502, the hybrid predicting section 64 receives thepredicted image block obtained by using the inter-frame prediction andthe predicted image block obtained by using the intra-frame pixel valueprediction from the motion compensating section 63 and the spatialpredicting section 74, respectively.

[0309] In step 1503, the hybrid predicting section 64 calculates thefirst SAD between the macro block sent by the inputting section 61 andthe predicted image block obtained by using the inter-frame prediction,and the second SAD between the macro block sent by the inputting section61 and the predicted image block obtained by using the intra-frame pixelvalue prediction.

[0310] In stop 1504, the hybrid predicting section 64 compares theaforementioned first with second SADs for each sub-block. When the firstSAD is smaller, the hybrid predicting section 64 sets the predictionmode of the sub-block of interest to the “inter-frame prediction mode”in step 1505. When the second SAD is smaller, the hybrid predictingsection 64 sets the prediction mode of the sub-block of interest to the“intra-frame pixel value prediction mode” in step 1506.

[0311] In step 1507, the hybrid predicting section 64 judges whether ornot the selection of the prediction mode is performed for everysub-block within the macro block of interest. When the selection is notperformed for every sub-block, the hybrid predicting section 64 returnsto the step 1502 and performs the selection of the prediction mode forthe rest of the sub-blocks.

[0312] When the selection of the prediction mode is performed for everysub-block, the hybrid predicting section 64 judges whether or not theaforementioned selection is performed for every macro block mode (INTER16×16 mode, etc.) in the macro block of interest in step 1508. When theaforementioned selection is not performed for every macro block mode,the hybrid predicting section 64 returns to the step 1501 and performsthe aforementioned selection of the prediction mode for the rest of themacro block modes.

[0313] When the selection of the prediction mode is performed for everysub-block and every macro block mode, the hybrid predicting section 64compares the sums of the SADs for the entire macro block of interest inthe respective macro block modes in step 1509.

[0314] Moreover, the hybrid predicting section 64 also compares the sumsof the SADs with the SAD in the case of performing the intra-frame pixelvalue prediction for the entire macro block (16×16 pixels) by thespatial predicting section 74.

[0315] The hybrid predicting section 64 applies the “macro block mode”providing the smallest SAD as the “macro block mode” of the macro blockof interest.

[0316] In step 1510, the hybrid predicting section 64 sends this “macroblock mode” to the variable length encoding section 67 and the hybridprediction decoding section 71.

[0317] Herein, when the “INTER mode” including a sub-block of the“intra-frame prediction mode” is selected as the “macro block mode”, thehybrid predicting section 64 sends the “macro block mode”, the“prediction mode selection states” of the individual sub-blocks, and the“motion vector”, or “pixel value prediction methods”, to the variablelength encoding section 67 and the hybrid prediction decoding section71.

[0318] The video coding transmission system according to this embodimenthas been described as the configuration compliant with ITU-T H.26L.However, the present invention is not limited to this configuration andapplicable to various video coding transmission systems which perform acoding process in units equivalent to the macro blocks (first imageblocks) using the prediction modes of the “inter-frame prediction mode(first prediction mode)” and the “intra-frame prediction mode (secondprediction mode)”.

[0319] Moreover, the method of discriminating the “inter-frameprediction mode” and the “intra-frame prediction mode” for eachsub-block is not limited to the configuration shown in this embodimentand employs various syntaxes indicating which of the “inter-frameprediction mode” or the “intra-frame prediction mode is applied to eachsub-block (second image block) within the macro block (first imageblock)”.

[0320] It is conceivable that the method of discriminating the“inter-frame prediction mode” and the “intra-frame prediction mode” foreach sub-block is multiplexed with various pieces of informationtransmitted in sub-blocks and transmitted like the “Ref_frame” in thisembodiment.

[0321] For example, when the sub-block of interest is set to the“intra-frame prediction mode”, the method can be indicated by a specialvector value or difference vector value using the motion vectordifference information (MVD).

[0322] Moreover, the information for discriminating the “inter-frameprediction mode” and the “intra-frame prediction mode” may betransmitted separately from the “Ref_frame” and the “MVD” by definingnew information to be transmitted in sub-blocks.

[0323] In addition, it can be designed to set a special code, namely,the escape code to the coding code of the “MVD” and to indicate that theprediction mode is the “intra-frame prediction mode” when this code istransmitted.

[0324] Note that, in this embodiment, there could be some cases wherethe “intra-frame prediction mode” is selected for every sub-block whenthe “INTER mode” is selected as the macro block mode.

[0325] The coding result in such a case is not necessarily the same asthat in the case where the “INTRA mode” is selected as the macro blockmode because of the different information such as the, “pixel valueprediction method”.

[0326] Therefore, the presence of such a “macro block mode” may bepermitted.

[0327] However, in this case, there are a plurality of macro block modesusing intra-frame prediction with no variation in the prediction method(motion compensated inter-frame prediction or intra-frame pixel valueprediction).

[0328] In order to prevent such a situation, the presence of theplurality of macro block modes using the intra-frame prediction may beavoided by forbidding the “INTER mode” in which every sub-block is the“intra-frame prediction mode” or by forbidding the “INTRA mode” as the“macro block mode”.

FUNCTION AND EFFECTS OF VIDEO CODING TRANSMISSION SYSTEM ACCORDING TOEMBODIMENT 3

[0329] With the video coding transmission system according to theEmbodiment 3, the hybrid predicting section 64 can optimally generatethe predictive residual signals in which the spatial redundancy or thetemporal redundancy is reduced from the sub-blocks. Accordingly, theefficient coding can be performed even in the case where the portionwhich is desirable to be encoded with the inter-frame prediction modeand the portion which is desirable to be encoded with the intra-frameprediction mode are mixed within a macro block.

[0330] (Others)

[0331] A program causing a computer 100 to execute functions of theencoders 20 and 60 and the decoders 40 and 80 according to theaforementioned Embodiments 1 to 3 can be recorded in a computer-readablerecording medium. As shown in FIG. 19, this computer-readable recordingmedium is, for example, a floppy disk 101, a compact disk 102, an ICchip 103, a cassette tape 104, or the like. This computer readablerecording medium with the program recorded can facilitate retention,delivery, sales, and the like of the program.

[0332] Industrial Applicability

[0333] As described above, according to the present invention, it ispossible to allow the portion which is subjected to coding with theinter-frame prediction mode and the portion which is subjected to theintra-frame prediction mode to be mixed in one macro block by switchingthe prediction modes without changing the frame work of macro blocks.

1. An encoder for encoding each first image block of a video,comprising: prediction mode selection information generating means forgenerating prediction mode selection information which indicates that afirst prediction mode for reducing temporal redundancy is applied toeach of second image blocks or that a second prediction mode forreducing spatial redundancy is applied to each of the second imageblocks, the second image blocks being obtained by dividing the firstimage block; predictive residual signal generating means for generatinga predictive residual signal by applying the selected first or secondprediction mode to each of the second image blocks; and transmittingmeans for encoding and transmitting the prediction mode selectioninformation and the predictive residual signal.
 2. The encoder accordingto claim 1, wherein the transmitting means encodes and transmitsprediction mode related information necessary for a coding process inthe first or second prediction mode.
 3. The encoder according to claim2, wherein the predictive residual signal generating means generates thepredictive residual signal from each of the second image blocks bymotion compensated inter-frame prediction using a motion vector, whenthe first prediction mode is selected to be applied, and the predictiveresidual signal generating means generates the predictive residualsignal from each of the second image blocks by intra-frame prediction,when the second prediction mode is selected to be applied.
 4. Theencoder according to claim 3, wherein the transmitting means encodes andtransmits information indicating the motion vector as the predictionmode related information, when the first prediction mode is selected tobe applied.
 5. The encoder according to claim 3, wherein the predictiveresidual signal generating means generates the predictive residualsignal by motion compensated inter-frame prediction using a same motionvector within the first image block.
 6. The encoder according to claim4, wherein the transmitting means transmits the prediction modeselection information before the prediction mode related information. 7.The encoder according to claim 2, wherein the predictive residual signalgenerating means generates the predictive residual signal from each ofthe second image blocks by a pixel value prediction method using a valueof a pixel adjacent to each of the second image blocks, when the secondprediction mode is selected to be applied, and the transmitting meansencodes and transmits the pixel value prediction method as theprediction mode related information.
 8. The encoder according to claim2, wherein the transmitting means encodes and transmits the predictionmode related information in association with the prediction modeselection information.
 9. The encoder according to claim 2, wherein thetransmitting means does not transmit the prediction mode relatedinformation of the first prediction mode, when the first prediction modeis not applied to each of the second image blocks.
 10. A decoderdecoding a video, comprising: prediction mode selection informationdecoding means for decoding prediction mode selection information whichindicates that a first prediction mode for reducing temporal redundancyis applied to each of second image blocks or that a second predictionmode for reducing spatial redundancy is applied to each of the secondimage blocks, the second image blocks being obtained by dividing a firstimage block, the first blocks being obtained by dividing the video; andvideo decoding means for decoding each of the second image blocks of thevideo, based on the first or second prediction mode selected by theprediction mode selection information.
 11. The decoder according toclaim 10, further comprising prediction mode related informationdecoding means for decoding prediction mode related informationnecessary for a coding process in the first or second prediction mode,and wherein the video decoding means decodes the video using theprediction mode related information.
 12. The decoder according to claim11, wherein the video decoding means decodes each of the second imageblocks of the video from the predictive residual signal by motioncompensated inter-frame prediction, when the first prediction mode isselected to be applied by the prediction mode selection information, andthe video decoding means decodes each of the second image blocks of thevideo from the predictive residual signal by intra-frame prediction,when the second prediction mode is selected to be applied by theprediction mode selection information.
 13. The decoder according toclaim 12, wherein the prediction mode related information decoding meansdecodes information indicating a motion vector as the prediction moderelated information, when the first prediction mode is selected to beapplied by the prediction mode selection information.
 14. The decoderaccording to claim 12, wherein the prediction mode selection informationdecoding means decodes the prediction mode selection information beforethe prediction mode related information.
 15. The decoder according toclaim 11, wherein the video decoding means decodes the video usingmotion compensated inter-frame prediction by a same motion vector withinthe first image block, when the first prediction mode is selected to beapplied by the prediction mode selection information.
 16. The decoderaccording to claim 11, wherein the prediction mode related informationdecoding means decodes a pixel value prediction method related to thesecond image block as the prediction mode related information, when thesecond prediction mode is selected to be applied by the prediction modeselection information, and the video decoding means decodes the videousing the pixel value prediction method.
 17. The decoder according toclaim 10, wherein the prediction mode selection information decodingmeans decodes the prediction mode selection information of each of thesecond image blocks.
 18. The decoder according to claim 11, wherein theprediction mode selection information is encoded in association with theprediction mode related information.
 19. The decoder according to claim11, wherein the prediction mode related information decoding means doesnot decode the prediction mode related information of the firstprediction mode, when the prediction mode selection informationindicates that the first prediction mode is not applied to each of thesecond image blocks.
 20. An encoding method of encoding each of firstimage blocks of a video, comprising: a step A of generating predictionmode selection information which indicates that a first prediction modefor reducing temporal redundancy is applied to each of second imageblocks or that a second prediction mode for reducing spatial predictionmode is applied to each of the second image blocks, the second imageblocks being obtained by dividing a first image block, the first imageblocks being obtained by dividing the video; a step B of generating apredictive residual signal by applying the selected first or secondprediction mode to each of the second image blocks; and a step C ofencoding and transmitting the prediction mode selection information andthe predictive residual signal.
 21. The encoding method according toclaim 20, wherein, in the step C, prediction mode related informationnecessary for a coding process in the first or second prediction mode isencoded and transmitted.
 22. The encoding method according to claim 20,wherein, in the step B, the predictive residual signal is generated fromthe each of the second image blocks by motion compensated inter-frameprediction using a motion vector, when the first prediction mode isselected to be applied, and in the step B, the predictive residualsignal is generated from the each of the second image blocks by theinter-frame prediction, when the second prediction mode is selected tobe applied.
 23. The encoding method according to claim 22, wherein, inthe step C, information indicating the motion vector is encoded andtransmitted as the prediction mode related information, when the firstprediction mode is selected to be applied.
 24. The encoding methodaccording to claim 22, wherein, in the step B, the predictive residualsignal is generated by motion compensated inter-frame prediction using asame motion vector within the first image blocks.
 25. The encodingmethod according to claim 23, wherein, in the step C, the predictionmode selection information is transmitted before the prediction moderelated information.
 26. The encoding method according to claim 21,wherein, in the step B, the predictive residual signal is generated fromeach of the second image blocks by a pixel value prediction method usinga value of a pixel adjacent to each of the second image blocks, when thesecond prediction mode is selected to be applied, and in the step C, thepixel value prediction method is encoded and transmitted as theprediction mode related information.
 27. The encoding method accordingto claim 21, wherein, in the step C, the prediction mode relatedinformation is encoded and transmitted in association with theprediction mode selection information.
 28. The encoding method accordingto claim 21, wherein, in the step C, the prediction mode relatedinformation of the first prediction mode is not transmitted, when thefirst prediction mode is not applied to each of the second image blocks.29. A decoding method for decoding a video, comprising: a step A ofdecoding prediction mode selection information which indicates that afirst prediction mode for reducing temporal redundancy is applied toeach of second image blocks or that a second prediction mode forreducing spatial prediction mode is applied to each of the second imageblocks, the second image blocks being obtained by dividing a first imageblock, the first image blocks being obtained by dividing the video; anda step B of decoding each of the second image blocks of the video, basedon the first or second prediction mode selected by the prediction modeselection information.
 30. The decoding method according to claim 29,wherein in the step A, prediction mode related information necessary fora coding process in the first or second prediction mode is decoded, andin the step B, the video is decoded using the prediction mode relatedinformation.
 31. The decoding method according to claim 30, wherein thevideo is decoded using motion compensated inter-frame prediction in thestep B, when the first prediction mode is selected to be applied by theprediction mode selection information in the step A, and the video isdecoded using intra-frame prediction in the step B, when the secondprediction mode is selected to be applied by the prediction modeselection information in the step A.
 32. The decoding method accordingto claim 31, wherein, in the step A, information indicating a motionvector is decoded as the prediction mode related information, when thefirst prediction mode is selected to be applied by the prediction modeselection information.
 33. The decoding method according to claim 30,wherein, in the step A, the prediction mode selection information isdecoded before the prediction mode related information.
 34. The decodingmethod according to claim 32, wherein, in the step B, the video isdecoded using motion compensated inter-frame prediction by a same motionvector within the first image block, when the first prediction mode isselected to be applied by the prediction mode selection information. 35.The decoding method according to claim 30, wherein in the step A, apixel value prediction method related to the second image block isdecoded as the prediction mode related information, when the secondprediction mode is selected to be applied by the prediction modeselection information, and in the step B, the video is decoded using thepixel value prediction method.
 36. The decoding method according toclaim 29, wherein, in the step A, the prediction mode selectioninformation of each of the second image blocks is decoded.
 37. Thedecoding method according to claim 30, wherein the prediction modeselection information is encoded in association with the prediction moderelated information.
 38. The decoding method according to claim 30,wherein, in the step A, the prediction mode related information of thefirst prediction mode is not decoded, when the prediction mode selectioninformation indicates that the first prediction mode is not applied toeach of the second image blocks.
 39. A program causing a computer tofunction as an encoder for encoding each first image block of a video,the program comprising: prediction mode selection information generatingmeans for generating prediction mode selection information whichindicates that a first prediction mode for reducing temporal redundancyis applied to each of second image blocks or that a second predictionmode for reducing spatial redundancy is applied to each of the secondimage blocks, the second image blocks being obtained by dividing thefirst image block; predictive residual signal generating means forgenerating a predictive residual signal by applying the selected firstor second prediction mode to each of the second image blocks; andtransmitting means for encoding and transmitting the prediction modeselection information and the predictive residual signal.
 40. A programcausing a computer to function as a decoder for decoding a video, theprogram comprising: prediction mode selection information decoding meansfor decoding prediction mode selection information which indicates thata first prediction mode for reducing temporal redundancy is applied toeach of second image blocks or that a second prediction mode forreducing spatial redundancy is applied to each of the second imageblocks, the second image blocks being obtained by dividing a first imageblock, the first image blocks being obtained by dividing the video; andvideo decoding means for decoding each of the second image blocks of thevideo, based on the first or second prediction mode selected by theprediction mode selection information.
 41. A video coding transmissionsystem having an encoder and a decoder, wherein: the encoder comprises:prediction mode selection information generating means for generatingprediction mode selection information which indicates that a firstprediction mode for reducing temporal redundancy is applied to each ofsecond image blocks or that a second prediction mode for reducingspatial redundancy is applied to each of the second image blocks, thesecond image blocks being obtained by dividing a first image block, thefirst image blocks being obtained by dividing the video; predictiveresidual signal generating means for generating a predictive residualsignal by applying the selected first or second prediction mode to eachof the second image blocks; and transmitting means for encoding andtransmitting the prediction mode selection information and thepredictive residual signal, and the decoder comprises: prediction modeselection information decoding means for decoding the prediction modeselection information of each of the second image blocks; and videodecoding means for decoding each of the second image blocks of thevideo, based on the first or second prediction mode selected by theprediction mode selection information.
 42. A video coding transmissionmethod comprising the steps of: encoding and transmitting predictionmode selection information and a predictive residual signal in a codingprocess, the prediction mode selection information indicating that afirst prediction mode for reducing temporal redundancy is applied toeach of second image blocks or that a second prediction mode forreducing spatial redundancy is applied to each of the second imageblocks, the predictive residual signal being generated by applying theselected first or second prediction mode to each of the second imageblocks, the second image blocks being obtained by dividing each of thefirst image blocks; decoding the prediction mode selection informationof each of the second image blocks in a decoding process; and decodingeach of the second image blocks of the video, based on the first orsecond prediction mode selected by the prediction mode selectioninformation.